home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
United Public Domain Gold 2
/
United Public Domain Gold 2.iso
/
utilities
/
pu652.dms
/
pu652.adf
/
AIBB3.docs
< prev
next >
Wrap
Text File
|
1994-11-20
|
59KB
|
952 lines
Amiga Intuition Based Benchmarks (AIBB)
A System Performance Evaluation Utility for the Commodore Amiga
Release Version 3.0
This program's primary purpose is to time-test low-level performance
of an Amiga system, and compare the results with known figures for 3 other
machines, namely the Amiga 2000 (or 500), the Amiga 2500/20 (68020 based
version), and the Amiga 3000/25 (25Mhz model). The comparisons made within
the program are designed to be a 'General' type of comparison. They are far
from 'exact to 0.000000000xx percentage, and do not always reflect
real-world applications performance. They can give an idea of the low-level
performance marks of a particular system, however. Because of the nature of
computer systems, and the fact that differences can occur between two machines
of the same type, the comparisons made should be used as an average view of
system performance. Many comparison samples were used to gather these figures,
and most should come within a good proximity of the values. The conditions
under which these values were obtained is explained in the documentation
below. If you want to REALLY be sure of the comparisons, ideally you should
run the tests on your machine, record the time value each test displays
(the run-time for each test), and do the same thing to the machine you wish
to compare against, and evaluate the differences. This program will give
you an average result answer however, allowing you to make general comparisons
with other types of machines and configurations Please do read the
subsection under menus describing the 'Be Selfish' option...there are some
warnings you MUST read there, and important information pertaining to the
comparision evaluations with this option.
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Amiga, all Amiga models, and Intuition are trademarks of Commodore-Amiga Inc.
MC68000, MC68010, MC68020, MC68030, and MC68040 microprocessor designations,
M68851 Memory Management Unit, and M68881, M68882 Floating Point coprocessor
designations are trademarks of Motorola Inc.
Disclaimer: I make no guarantees that this program will be accurate. I
have made every attempt to make it so, but I cannot be held
responsible for its accuracy, and/or consequences of such.
(In other words, include standard disclaimer here :-) )
Also, should this program maim and/or destroy various aspects
of your Amiga, I am also not responsible. (Well...it hasn't
killed yet...): Another meaning; if you run this program, and
something about it surprises you, causing you to
spill coffee into your machine, which sends a power spike
feedback through the power lines, down to your local power
station, which subsequently explodes in flames...,
I don't, and never did exist. ;^)
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
PROGRAM REQUIREMENTS:
This program can be used on any Amiga...but there are a few requirements
to use all of it's features. In order to use the Log File saving utility,
you must have the req.library in you libs: directory (v2.5 or higher). Please
read the appropriate sections below for more information on this topic.
Note that this requirement is not in actuality a "requirement", but rather
simply necessary to use a certain feature. Special Note: Under the CLI AIBB
will create it's own stack, when running AIBB from the workbench, however,
you must be sure the STACK field of the icon used is set to the value
of 18500. The supplied icon is already pre-set with this value. The only
errors AIBB may encounter which will result in the program aborting before
starting are that: 1.)If you started from the WorkBench, and the stack
field in the icon was less than 18500 (CLI users need not worry about any
settings), or 2.)AIBB was unable to open either the intuition.library,
graphics.library, or dos.library (all of which are KickStart libraries,
which is in ROM on most machines). If AIBB encounters any of these problems,
it will simply abort gracefully, perhaps flashing the screen.
******************************************************************************
******************************************************************************
******************************************************************************
USING THE PROGRAM:
This program was designed to be fairly easy to use...which is the
reason I utilized an Intuition interface. Basically, test options and
such are selected from menu items, and a test is intiated by selecting
it's gadget on the screen. Again, note that the comparisons made in this
program are meant to be general. Here's an overview:
*****************************************************************************
************************* The Main Screen ***********************************
*****************************************************************************
The screen is divided into two areas. On the left side, you will find
the performance graph, and CPU status indications. The performance graph is
basically just a graphical display of the percentage performance of the
various machines, including your own. The scale represents percentages in
hundreds, i.e. a factor of 6 would be 600%, etc... If for some reason your
machine measures off the scale which is currently displayed, AIBB will
auto-scale the graph to a suitable factor of the base scale (0-14) to fit
the data on the screen. When a test is performed which is much less than
a currently displayed scale, then AIBB will auto-scale downward (the limit
being the base of 0-14) The CPU status area is directly below the graph,
and shows the CPU type in your system, and the status of the various caches,
if applicable, as well as if burst mode on the caches has been enabled. If
your CPU does not have a particular cache feature, the display will show
N/A (Not Applicable). The approximate clockspeed of your CPU is also
displayed in this area (evaluated when AIBB is invoked), as well as the FPU
type on the system, should one exist. [NOTE: These are APPROXIMATE
clockspeed indications...+ or - 5-10%]
The right side of the display contains the numerical analysis display
boxes and the test selection gadgets. From top to bottom, the features are:
Benchmark Result: This box shows the running time in seconds for the
benchmark performed. It is not a comparison, but
simply a display of the time used in performing the
benchmark.
Base Machine = : This area will show what machine is currently being
used as the base machine for percentage calculations.
The next 4 boxes in decending order are the numerical representations of
the percentage performance of each machine for a particular benchmark.
All percentage calculations are percentages relative to the base
machine, where the base machine will always show 100% performance
(naturally, as it is relative to itself). In addition to the
"Base Machine = " indicator noted above, the Base Machine will have
the marking for it's percentage box highlighted. Also displayed with
the performance percentages in the boxes is an abbreviation for the
type of test code being used for the comparison...this is explained
further under the Menus section (Test Option and Comparison Options).
------------------------------------------------------------------------------
The lower right hand side of the screen contains the test gadgets.
selecting a given gadget invokes the test it represents. AIBB currently
implements 12 tests. These tests are divided into two types: "Normal"
tests, and Floating Point tests. "Normal" tests are ones which do not
utilize any floating-point operations, utilizing integer math only, or
no math at all. "Normal" Tests are indicated by dark blue labels upon
their gadgets, whilst Floating Point tests are indicated by bright blue
lables. A description of the tests follows, along with the test's memory
usage. This may be important in determining where the test will end up
Allocating it's memory. Note that tests which do not allocate external
memory are using the program's stack to store variables, and miscellaneous
items. A program's stack is an area of memory created upon the task's
invocation. Under many circumstances, you may have control of this
stack size...for example, program's invoked under the CLI or Shell will
often inherit the default stack set for that CLI/Shell window...which
can be set/changed with the c: "Stack" command. However, AIBB does not
inherit this value, instead allocating it's own stack area (approximately
18K) in the case of the CLI... or having it set higher in the case of
WorkBench invocation. The tests:
WritePixel: (NORMAL test)
The WritePixel Benchmark will open a small window on the screen
and fill it with a box of a given color. The filling is done one
pixel at a time, utilizing the graphics.library (OS) routines
SetAPen() [set the current RastPort primary pen color] and WritePixel()
[which sets a pixel to the given primary pen color]. The test is
basically a benchmark of the time needed to call these routines, and
for them to execute. The loop time is factored out of the test
result. For the most part, this test will be useful for evaluating
the effective ROM image access time for systems which differ from
the conventional ROM access method found on the Amiga 2000, namely
accessing the ROM over a 16 bit bus. Many accelerators are able
to utilize various methods (most commonly through Dave Haynie's
SetCPU utility) to move the image of the ROM code into a faster
memory environment to take advantage of the additional speed and
wider bus bandwidth of higher 680x0 processors (68020, 68030, etc).
This commonly involves utilizing a Memory Management Unit (MMU)
[68551, external device anchored to the 68020, or internally implemented
on the 68030] to divert ROM routine calls to a copy of the ROM
image made in the faster (32-bit) memory. The time needed to
access and execute this image is dependent upon the speed of the
processor/memory setup. [Of course, on the Amiga 3000, the bus is
nominally 32-bits wide, therefore no movement of the ROM image is
required]. Now, there's more to this...accelerated Amigas still
have to access CHIP RAM for this test, as graphics functions are
performed. Therefore, this test will also show (somewhat) the
efficiency of the accelerator (if present) when synchronizing to
and accessing the CHIP RAM bus. [Again, for the Amiga 3000, this
is not entirely applicable, as the design is integrated].
Now, this CHIP RAM bus->accelerator/processor interface efficiency
can cause differences in systems which are based upon similar
accelerated processors, (and at the same processor clockspeed).
Usually the difference is not too great, but it can occasionally
show. Now, for accelerators which do not do ROM image translations,
(usually for ones incapable of them), this test will mostly just
reflect the faster call/return to the benchmark code. However,
if an accelerator includes a cache of any sort, then the cache will
speed the routines themselves up somewhat, as the code for them
may be partially, or even fully cached by the accelerator, negating
the need for the bus access to utilize the OS routine.
Memory Usage: No memory external to AIBB is allocated.
Dhrystone: (NORMAL Test)
This test should be fairly familiar to most people, as it has
been utilized on many different system for benchmarking purposes.
It's a test which attempts to achieve a close-to-real-world-performance
evaluation of the system in question. It returns, not run-time in
seconds, but rather a rating of Dhrystones per second, where in this
case, the larger number indicates better performance.
Memory Usage: Minimal. 44 bytes external to AIBB are allocated.
Matrix: (NORMAL Test)
A matrix manipulation benchmark utilizing 3 50x50 integer matrices.
The test simply performs a series of operations (addition/subtraction,
multiplication, transposition, etc) upon these matrices. Now, the
test is set up in such a way that a great amount of time is spent
moving data, as well as performing arithmetic operations upon it.
Therefore, this could be thought of as also testing memory manipulation
efficiency. Also, the test is an indicator of how well a processor/
memory combination handles memory accesses to data, and operations on
such, as the test does not allow the processor to simply perform the
data operation within it's registers.
Memory Usage: 30000 bytes external to AIBB are allocated.
Fibonacci: (NORMAL Test)
The Fibonacci sequence is a mathematical sequence which has found
some application in the natural sciences, notably in the study of
the constructs of nature. The sequence is set up as follows:
x(0) = 0
x(1) = 1
...
x(j) = x(j) + x(j-1)
where x(j) is simply the next number in the sequence, read "x sub j".
Because of the relative simplicity of the sequence, it lends itself
easily to the creation of a recursive program algorithm to compute
the value. For those unfamiliar with programming in general, a
recursive function is one which calls itself from within itself.
Recursive algorithms in themselves are very inefficient, due to their
nature of repeatedly making a function call to themselves, which
involves repeated address offset calculations, stack value pushing,
etc... by the processor. However, it is sometimes interesting to use
such a function to test the efficiency of a system in it's handling of
multiple function calls. For this reason, I implemented the a
recursive Fibonacci calculation algorithm here, and the benchmark
consists of timing it when calculating the value of Fibonacci(24) 50
times. Each call to the recursive routine to perform the calculation
will result in 150,049 recursive function calls which the Fibonacci
routine will make to itself. For those curious, the value of
Fibonacci(24) is 46,368.
Memory Usage: No memory external to AIBB is allocated.
Sieve: (NORMAL Test)
Another test which should be familiar to most, the Sieve of
Erathosthenes. Its a fairly simple test for determining prime
numbers within a range of numbers. This test simply times your
system when implementing this algorithm, which is decribed fully
in many textbooks, or one can simply look at BYTE Magazine's
benchmarks, which use a similar Sieve test.
Memory Usage: No memory external to AIBB is allocated.
Sort: (NORMAL Test)
A series of 20,000 16-bit integers is sorted from a pseudo-random
setup, and the action is timed. "Pseudo-random" meaning that I did
not create the numbers in a random fashion, but rather in a mixed fashion
so that on each invocation of the test the numbers will be created in
the SAME mixed fashion. This is because the sorting algorithm is
sensitive to the mixing, and if each time the test was run a different
group of values was used, no two tests could be compared well. The
mixing method I used was to insure that the algorithm would be forced
to do the most work for each test.
Memory Usage: 40000 bytes are allocated external to AIBB.
LLines: (NORMAL Test)
This is a low-level test of the Amiga's hardware. It uses the
Blitter to draw lines (left to right, creating a box) across the
screen, in each of 16 colors...ending with black. This test
is similar to the WritePixel test in that it will hit the CHIP RAM
bus a great deal (more so in this case), but it makes no calls to
operating system routines, but rather the system's hardware.
[This is done in a "friendly, legal" fashion, so fear not]. Now, as
all Amiga's have the same basic hardware setup at this level, there
won't be many differences in benchmark time caused by this. At
the time of this writing, therefore, there is not a great deal of
use to this test. However, in the future, there may be other Amiga
systems which utilize a different bus or chip set for the display
hardware, so this test may become useful at that time. Small
differences noted for this test between the comparision machines
are primarily due to loop time differences between the machines
which could not be completely factored out.
Memory Usage: No memory external to AIBB is allocated.
IMath: (NORMAL Test)
Integer Math. This test performs a wide variety of integer math
functions. Included in these functions are the "normal" functions,
such as addition, subtraction, multiplication, division, and a
few additional bitwise functions, such as ANDing, ORing, and XORing.
Memory Usage: No memory external to AIBB is allocated.
FMath: (FLOATING POINT Test)
Floating Point Math. Similar to the IMath test, with the exeception
that Floating Point values and operations are utilized. With this test,
no bitwise operations are performed. Single precision floating point
operations/values are used here.
Memory Usage: No memory external to AIBB is allocated.
Savage: (FLOATING POINT Test)
This is another of the "probably familiar" tests. It is a standard
implementation of the Savage test, which makes nested calls to
trigonometric and other more complex functions to create a single
value. Double precision floating point operations/values are used.
Memory Usage: No memory external to AIBB is allocated.
FMatrix: (FLOATING POINT Test)
The FMatrix test is similar in concept to the Integer Matrix test
outlined above. Again, a great deal of data movement is performed,
in addition to the operations involved, which are floating point
operations in this case. With the matrix operations, the results
under Floating Point coprocessor equipped systems can be interesting
to note, as the system is not able to keep the data within speed-ready
registers, and thus must make many bus accesses for the data it needs.
Double-precision floating point math is used for this test.
Memory Usage: 37,500 bytes externally allocated.
BeachBall: (FLOATING POINT Test)
The longest test of the group. The BeachBall test was originally
written by Bruce Holloway of Weitek, and published in the March 1988
issue of Byte Magazine. It is essentially a very math-intensive
operation which basically draws a beachball on the screen, complete
with shading. The test opens a 640x400 interlaced 16-color screen,
and proceeds to render the picture. This test is closer to a true
"application" test, in that it actually does something visible, and
produces an output. The system will end up being tested in both the
floating point arena, AND in screen output performance, which is
done through traditional OS graphics handling calls [thus will be
affected by the speed of which, which in turn can be affected by
ROM re-mapping, etc]. The picture will remain on-screen for 5 seconds
after the rendering is complete to allow you to view the full creation.
Memory Usage: No memory external to AIBB is allocated.
TEST AND COMPARISON NOTES:
These tests are far from ideal. They suffer from traditional problems
related to benchmarks, in that they do not in and of themselves provide
a total picture of system throughput. Many factors must be taken into
account when evaluating system throughput, including the architecture in
question, it's characteristics, and it's slowest and fastest attributes.
The tests CAN provide a starting point for evaluating a system, however.
I urge you to look at them as such, rather than the complete answer as to
which system is fastest...that decision is more based on what a particular
individual's NEEDs are. [eg, if a person needs to do data transfer as a
major portion of their work, then perhaps a fast disk subsystem and
communications setup would be more appropriate to test than low-level
system performance. If applications are to be run which are very disk
intensive, then a faster disk subsystem will probably make more of a
difference towards system througput then a faster CPU/FPU combination...
all of these things must be taken into account]
Note that the tests CANNOT be aborted once begun (short of re-setting the
system). This is necessary as to provide such an abort mechanism would
require that I add code to poll the message ports to AIBB's interface...
this in itself would affect the benchmark as the time necessary to poll
those ports would show up within the benchmark time, and thus a true
benchmark result would not be possible. I warn you of this for a very
important reason: Some of AIBB's tests are _LONG_...especially on more
or less stock systems. In fact, if you are running a stock 68000-based
system, then most of the floating point tests will be excruciatingly slow.
[The Savage and BeachBall tests are the type where you go out for a while
when running them]. On faster systems this is not such a problem, but
(for example), on a 25MHz 68030 system in 32-bit SRAM (1 wait state), the
Savage will still take 80 or so seconds when NOT using a math coprocessor.
[Don't ASK where this came from... :^) ...Using the Coprocessor version of
this test, with a 25MHz 68882, the test will run in under 3 seconds].
Please do keep these things in mind if you run these tests...especially under
the "Be Selfish" option (explained further below)...the system has not
crashed, or hung on you....the test you selected may simply be VERY LONG,
depending on your system configuration and the type of test parameters you
selected...I have thoroughly tested AIBB...even under low memory situations,
and in the final incarnation of this version I have not been able to crash
it yet. If it should do so, however, I would appreciate being told about
it.]
******************************************************************************
****************************** Menu Items ************************************
******************************************************************************
The menu fields in the menu bar for AIBB are as explained below
==============================================================================
The GENERAL menu:
About:
This simply shows my neato little 'about the program and
life in general' requester. Fairly standard stuff.
-------------------------------------------------------------------------
System Status:
The screen will fade out and back in with a display of some key
notes of interest about your current system status. Various
elements of you system's current state and parameters will
be shown in the upper portion of the screen, including the
location of your system stack (this may be of importance, as is
explained later), Vector Base Register address (if present) and
the number of currently waiting and ready tasks are among the
items of interest. The lower portion of the display holds the
system's memory information. Each entry corresponds to a given
memory node in the system, such as your CHIP RAM area, a FAST RAM
board, etc. The current priority of each area's memory, it's
address, and number of free bytes are displayed, as well as
total memory figures current for the system. This information
may be useful for determining the exact conditions a test is
being run under.
-------------------------------------------------------------------------
Start/Stop Log File:
This will allow you to activate/deactivate AIBB's saving of
test results. [The menu will show 'Start' when you are
currently NOT saving results, and 'Stop' when you are]. When
first selected, AIBB will call up a file requester asking you
for the name of the Log file to use. Currently, the file
requester implemented is the one found in Colin Fox and Bruce
Dawson's req.library, which must be in your LIBS: directory in
order to access this feature. If AIBB cannot find this
shared library, the Log File menu option will be disabled
(ghosted), and you will not be able to save your results in
this manner....however the program will still function otherwise.
When a file is selected, AIBB first checks to see if it exists.
If not, it will create it, writing the identifier 'AIBBLogFile'
at the head of the file (standard text file). All future test
results will be appended to this file, until 'Stop Log File'
is selected, or the program exits. [Note: it is NOT necessary to
stop file logging before exiting...it is mainly there if you
wish to make another file]. If the file you select DOES exist,
AIBB will check the beginning of the file for the 'AIBBLogFile'
identifier...if it finds this, it will again append all results
to the end of this file (nothing will be overwritten)...if AIBB
does not find the identifier at the start of the file, a
requester will pop up warning that the file is NOT a standard
AIBB log file, and will ask you if you wish to cancel the
operation, OR use the file anyway. Be careful..this warning is
designed to keep you from writing over important files...or
programs. [NOTE: AIBB WILL happily write into a binary file if
told to continue...I will add some better checking in the future,
but for now, WATCH the warning!]. Note that AIBB does not buffer
results...it writes to the file as soon as a test is completed.
This shouldn't be too much of a problem, but if people find it
irritating, I will of course add in some buffering. If desired,
RAM: or a recoverable RAMdisk can be used to save results, but
care should be taken that the space the file takes will not use
up your system memory to an extent that the test results will be
affected. (eg, using up your FAST RAM or enough of it that the
memory for a test ends up being allocated in CHIP or SLOW-FAST
RAM, or another slower medium than the ideal you wished to
examine.
-------------------------------------------------------------------------
Quit:
Self explanatory.
The machine will be placed back into the state it was in when
the program was first invoked...regardless of the manipulations
which may have been done during the tests (See the section
on 680x0 cache manipulation).
==============================================================================
The TEST OPTIONS (second) Menu: This menu contains items relevant to the
tests being performed, and the way the results are displayed:
Be Selfish:
This is an important item. Basically, when this is
selected, the program will make sure no other tasks
are given CPU time while one of the tests is running.
In this way, you are assured that the test results will
reflect true CPU performance, and not lag due to other
tasks taking up some of the CPU's time. All of the
comparison figures for the other machines were obtained
with this option ON (A check will be placed by the
option when it is active), *******IF YOU WISH TO COMPARE
TO THE OTHER MACHINES IN THE PROGRAM MOST ACCURATELY,
HAVE 'Be Selfish' SELECTED****** Read on...When 'Be
Selfish' is active, the system will not respond to you
while a test is running. Meaning, you will not be able
to move your mouse pointer, and that most enjoyable
floppy drive 'clicking' will even stop. THIS IS NORMAL!
You will regain control when the test completes...the
system did not hang. Some tests do take a little time,
so hang in there. The only time you will really not
want to have 'Be Selfish' active is if you wish to
test the performance drain on your system from having
multiple tasks running. (or just to see what the normal
load of tasks does to your CPU performance). Now...you must
know something about this option. What AIBB is doing when
this option is active is invoking a Disable() call a bit before
each test, and then an Enable() afterwards. Disable()
is an operating system function which stops all interrupts
within the system. Since the Amiga's multitasking depends upon
interrupts for task switching operatons, this is also stopped,
and AIBB becomes the only task doing anything (there are
exceptions...that is explained later). ALL this aside, I MUST
give you a warning. I designed this program to test CPU/FPU
performance, and as such it is a good idea to have as little
running in the background as possible...aside from performance
issues though, there are some programs/operations which are very
timing sensitive. The Amiga's Multitasking Executive was
designed to be as close to real-time responsive as possible. As
such, programs and tasks under it may be sensitive to system
timing. Such programs may become erratic, or even crash if
a function such as Disable() is invoked for any length of time.
[Disable() should normally only be for a _brief_ time...this
program makes an exception of that in order to get more accurate
timing results]. Now, most programs I have encountered do not
have a problem with AIBB, but things to watch out for are
programs making heavy use of communications ports (serial,
parallel, etc, and disk-intensive programs. Now, this is not
to say AIBB is just waiting to crash your system...I've not had
a crash related to this yet. However, AIBB was designed basically
to be run ALONE, at least while in "Be Selfish" mode, and there
is basically little reason to run it otherwise while using this
mode. But do consider disk operations while in Be-Selfish mode,
and running a test to be a no-no. NOTE: AIBB's own file saving
is done in a safe way, and CAN be used with 'Be Selfish', as the
program only accesses the disk when not performing a test. If
you have AIBB just sitting there not doing anything, then you
do not have to worry, even with "Be Selfish" selected, as this
option is only effective while a test is in progress.
AIBB defaults to NOT using Be Selfish so that the quick-to-jump-in
person will not feel my program has locked-up the system when
they go to do a test. The first time you select 'Be Selfish'
from the menu, a requester will pop up reminding you of what
I just said here. Once the requester has appeared the
first time, it will not appear again during the course of the
program's execution, regardless of how you set the Be Selfish
menu option (otherwise it could become quite annoying :-) )
Again, with 'Be Selfish' NOT enabled, there are no restrictions
to this program's use, but again you may show a marked
performance drop, depending on your system's task load at the test
time.
---------------------------------------------------------------------------
Change Base:
This menu item allows the selection of the machine to be
used as the base in the percentage comparisons. The
submenu which appears next to it allows you to choose
which machine you wish to use as the base machine...the
default being the A2000. With any base, the
percentages, and the graph show the performance of both
your machine, and the other machines as compared to said
base. The base machine will always show itself to have
a relative performance of 100% (naturally, as it's
evaluating itself against itself). The other machines
will show percentages relative to the base machine's
performance.
---------------------------------------------------------------------------
Standard Test Options:
This menu option contains a submenu for selecting the
type of code to run for the 'Standard' (non-floating point)
tests (Dark Blue gadget lettering). It presently contains
two options:
Standard Code: (NCode)
This version of a particular test is
using normal code which will run on
any type of 680x0 processor, and thus
any Amiga. This is the default setting.
020/030 Code: (0XCode)
Tests run under this option use code
which is optimized to run more efficiently
under 68020,68030, and above processors.
It takes advantage of timing differences,
and specialized instructions present in
these processors. This option allows
you to run a test 'set up' to suit a
higher-level processor.
Both versions of the test perform the exact same operations,
except the machine code used to do it differs. This allows
comparisions to be taken with regard to more 'optimum' code
conditions for a particular system setup. (For example,
comparing how well your system performs a Matrix manipulation
while running 020/030 code, compared to 'Standard' code).
Note: if your system is a 68010 or 68000, you will not be
able to use the 020/030 code versions of the tests, as this
code will not run on those processors. Thus, when AIBB
determines the processor type on your system, it will disable
the selection if you do not have an 020 or above.
---------------------------------------------------------------------------
FP Test Options:
This menu item and its corresponding submenu are similar
in concept to the "Standard Test Options" menu item. The
difference here is that the selections made here affect the
Floating Point tests (Light Blue gadget lettering). Again,
two options are present:
Processor Math: (PCode)
If this option is used, AIBB will
perform any of the floating point tests
initiated using the main system processor.
(eg, all math will be done by the CPU)
Coprocessor Math: (CPCode)
This option causes AIBB to use
the version of a test which does the
floating point operations using a
math coprocessor (68881, 68882, or
on board the 68040).
Again, the tests do the same exact operations for either
version...simply the method used to carry out the operations
differs, in this case by the use of in-line coprocessor
instructions for the Coprocessor Math selection. Also, I
should note that the Coprocessor Math option is also utilizing
020/030 based code, as in the "Standard Test Options", as
the in-line instructions here require a standard interface to
the coprocessor, and thus a 68020 or above is also a requirement.
While true that software traps could be implemented to use
a coprocessor on a 68000 or 68010, this has not been used
much on the Amiga, and thus I felt that I would also use
020/030 code to allow a view of how well a system performed
floating point operations when completely set up to take
advantage of an 020/030 + coprocessor configuration. Therefore,
if you have an accelerator using a 68000 or 68010 which
utilizes F-line software traps to utilize a math coprocessor,
the code will most likely crash, as it will also be attempting
to use 020/030 instructions with your CPU. Now, to date I have
not seen any boards which are using the above said method
[effectively], so this probably will not be a problem.
In addition, AIBB tests the system for an FPU when it
is invoked, and if none is detected, the Coprocessor Math
selection will be disabled.
==============================================================================
COMPARISON OPTIONS MENU: The options present here are very related to
the last two options ("Standard Test Options" and "FP Test Options") present
in the "Test Options" menu. These options affect the comparision figures
used when the percentage performance evaluation is done agains the
comparison machines.
Cmp Standard Options: The options listed in the submenu here are
identical to the ones in the "Test Options"
item "Standard Test Options". However, this
does not change the type of code run by your
machine. What it does is change the internal
comparision figures to reflect the performance
of the comparision machines with the option
selected. For example, if you select
"020/030 Code", then the figures used in the
comparisions will be the test times for the
comparision machines when running 020/030 code.
Cmp FP Options: Identical to the above, save that it affects the
performance figures used for comparision with respect
to the floating point tests.
These options may be a little confusing, but here's the idea behind them:
With this setup, you may choose to run a test using "Standard" code. Yet,
you have "020/030 Code" selected under this menu for the comparisions.
Therefore, you will be comparing how well your machine runs a test using
"Standard" code, with respect to the comparision machines running
the same test, but with an advantage: 020/030 code. Likewise, you may
disadvantage the comparison machines by leaving these options in their
"Standard" mode, and run the tests using 020/030 code for your machine
(Thus possibly showing a great percentage performance increase).
Likewise for the floating point tests and options.
==============================================================================
Depending on the processor your machine has, you may find another menu,
'Processor'. This field only appears if you are running a 68020 or above
and allows you to switch the caches of your processor on/off, to see the
effects doing this has on test results. The menu configures itself to
your processor type, so if you are running a 68020, the only item that will
appear under this menu is 'Switch I-Cache'. For higher processors, items
to manipulate the Data and Instruction cache, and the BURST modes will
also appear. When a cache is changed this way, the new cache status will
appear in the CPU status indication area explained below. If you do
choose to manipulate the caches during the tests, and get lazy and don't
set them back the way they were, the program will automatically set the
caches to the state they were in upon startup when you quit the program.
NOTE: As I indicated, if you are running a 68000 or 68010, this menu will
not be present, as it is of no use to those processors. Also, although the
choice for switching BURST mode on may appear for your machine's processor,
it may or may not have effect. This is because your system must be capable
of using this mode, and some are not. If you switch it on and your machine
isn't able to use it, no harm will come of it. In fact, AIBB will
still set the BURST enable bit(s) in the Cache Control Register, but the
processor will simply ignore them.
==============================================================================
COMPARISON NOTES: (IMPORTANT...PLEASE READ)
The machines used by AIBB for the comparisions had the following
configurations...all figures were acquired with the "Be Selfish" option
ENABLED. Please read these sections carefully so that you have a clear
picture of the machines/conditions which generated the comparison figures
AIBB uses internally.
A2000:
Numerous Amiga 2000s (and 500s, which are basically the same machine at
this level) were used to gather the figures for this set of comparison
values. The machines were all equipped with true FAST RAM, and all tests
were run out of this memory.
A2500/20:
This is the earlier Amiga 2500, the model equipped with the CBM A2620
68020-based accelerator. The 68020 on-board the accelerator is
clocked at 14.28MHz, as is the 68881 math coprocessor included. The
accelerator is a synchronous design. A full complement of 32-bit memory
(standard to the board) was used, and the tests run out of this memory.
The memory on the board was the standard 100ns memory at 2 wait states.
The A2620 can be utilized with 80ns memory at 1 wait state by setting
a appropriate jumper (and using the proper parts), and this should be
kept in mind if you are testing a system equipped with such. However,
I chose to use the 100ns parts @ 2 wait states as they were the
standard parts used with the board. The board does include a 68851
Memory Management Unit (MMU), so ROM translations on this machine are
possible, and were done using Dave Haynie's SetCPU (v1.6) utility.
SetCPU also moves the System Stack out of CHIP RAM, and into 32-bit
memory when possible, so this must be kept in mind. The instruction
cache on the 68020 was left ON (the OS will turn it on for you).
A3000/25:
The 25MHz model of the Amiga 3000 was used for the tests, which utilizes
a 25MHz 68030 CPU and 68882 FPU. The following configuration was
evident on the test systems:
2 megabytes of CHIP RAM, 8 megabytes of FAST. The FAST RAM on the
system was of the 80ns Static Column variety. This is important to note,
as the A3000 can be found in many memory configurations. The Static
Column RAM (SCRAM) allows the use of the 68030's BURST mode of data
access. This does result in a faster system. Normal DRAM will show
slower benchmark performance than the SCRAM memory model. The tests
were, of course, run out of the system's FAST RAM. The 68030 settings
were with Instruction and Data caches ON, and BURST mode for both
caches ENABLED.
AIBB's internal test figures were acquired under AmigaOS 1.3....there may be
differences under 2.0 (or other OS versions) which may cause differences with
the test results for your machine if you are using that version of the OS.
Particularly in question may be the OS-call heavy tests. [WritePixel
specifically]. Now, as of this writing, AmigaOS 2.0 was not yet burned into,
or shipping in ROM form yet. Systems running this OS version are using
RAM-resident versions of the KickStart image, which in and of itself will
affect the benchmark results. [eg, a 25MHz 030 system under 2.0 may not
compare the same as the equivalent system under 1.3, particularly for such
tests as the WritePixel, given that the 1.3 system will probably be running
OS code within ROM...[or perhaps mapped to 32-bit RAM...in either case there
may be differences]]. I fully intend to re-evaluate AIBB's comparisons when
2.0 is shipping in ROM form, and issue an appropriate update, if need be.
However, until that time, any comparisions made under 2.0 must be recognized
as being between a 2.0 equipped machine, and 1.3-equipped machines...which
may or may not provide significant differences.
All performance figures for the comparison machines are for NTSC
machines. PAL machines (or machines with the 1MB Agnus running PAL) will
may show false results due to the timing differences. If it becomes a
nuisance, I will add PAL support to the comparisons, as it wouldn't be
THAT big of a chore :^).
Again, comparisions made by AIBB are percentage based. The Graph, and
numerical boxes display the percentage performance of the system being
tested with respect to the "base" system. The base system will always
show a performance "base" of 100%...so, for example, if you run a test
and get 600%, then you are running 600% of the speed of the base system
on that test...this translates to 6 times the speed. The display boxes
for the numerical percentages will also display the type of code being
used for the comparisons...this is so you may keep track of just what
you are comparing against. The abbreviations are:
(Normal [Integer] tests)
NCode ---- Normal Code model
0XCode ---- 020/030 Code models
(Floating-Point tests)
PCode ---- Processor-based code model
CPCode ---- Coprocessor-based code model
Again, I emphasize that the comparision figures AIBB has housed
internally were acquired with the "Be Selfish" option ON. If this option
is not used, you will see significant differences with your performance
with respect to the comparision machines...even if you have an identical
machine. Also, it is important to keep track of where AIBB is getting
memory for it's tests on your system.
Tests can be affected greatly depending on where the memory ends up
being allocated, provided external memory is needed for a given test. The
Amiga utilizes a prioritized memory allocation scheme. Memory of higher
priority is given precedence for general (non-specific) allocation requests,
and if enough memory within a given node with a given priority exists (and
it is contiguous), that memory will be allocated over the same size memory
in a lower priority area. Now, if the memory in the higher priority area
is unavailable, the system will search downward through the areas (and
priorities) until it finds a suitably large, contiguous memory region. This
can be quite important to note on accelerated systems which have, in
addition to 32-bit memory on an accelerator, 16-bit memory. If AIBB requests
a memory block for a test, and there are no free areas in your 32-bit RAM
(or it is severely fragmented to the point that no contiguous chunks of the
required size exist), AIBB may very well end up with memory from the 16-bit
pool...and this will affect the benchmark result. [You may desire this, you
may not...but it is something to keep in mind]. The same argument exists
for CHIP or SLOW-FAST RAM vs. FAST RAM. [SLOW FAST RAM exists on Amigas
with the older 512K Agnus chip. It is the memory on the A501 expander for
the A500, and the other 512K out of 1-meg on the stock A2000 with this older
chip. This memory is still on the CHIP RAM bus, although it is not treated
AS CHIP RAM...therefore it is still subject to the bus contention found on
this bus between the processor and the custom chips (which may result in
performance degradation). A program supplied by CBM called "FastMemFirst"
is necessary and useful for giving any TRUE FAST RAM on the system priority
over the SLOW-FAST memory which may be present. People with the 1-meg Agnus
chip (or the A3000 for that matter) do not have SLOW-FAST memory, and do not
have to concern themselves with this.].
Certain utilities have surfaced which are designed to improve (to a
small degree) system performance on the Amiga. Two such programs are
MoveSSP, which moves the system Supervisor stack out of CHIP memory, where
it is normally located, into FAST RAM, in order to alleviate any small
amount of bus contention which may occur during accesses to this stack.
Another is VBR (or MoveVBR), which moves the system vector base to FAST RAM
from CHIP RAM..(this utility only functions on Amigas with a Vector Base
Register...68010 processors and above). It should be noted that none
of the comparison figures were obtained with any of these utilities in use.
They could cause a slight speedup for your system, (if you notice it at all),
so keep this in mind when benchmarking a system, and be sure to keep track
of just what your system configuration is at a given time for the tests.
[For the most part, all of the comparison figures were obtained with
"vanilla" systems, in the sense that little to nothing was added to the
base startup configuration]. Also see the note above on SetCPU v1.6's
movement of the System Stack...for the A2500/20 tests. This is the same
basic operation as with MoveSSP, (except the MMU is used) so systems running
SetCPU v1.6 will have this stack translated, unless explicitly disallowed.
So, the basic rundown is that SetCPU v1.6 WAS used with stack translation for
the A2500/20...but nothing special was done for the A2000 or A3000/25. I
left the stack translation into the A2500/20 setup simply because it is
the default method in SetCPU v1.6 of translating the system ROM into
32-bit memory.
==============================================================================
A WORD ON THE BENCHMARKS:
The benchmarks employed in AIBB, as I have stated, are not in any way
ideal. They are meant primarily to give a general feel for the performance
of the system in question. However I do wish to make a few comments about
the benchmark code. The code for the benchmarks used here is not optimized
heavily. This is a deliberate action, and with purpose. Code optimization
by compilers (and assemblers to a different extent) can be a wonderful tool
for enhancing program performance. However, it can also make _system_
benchmarks quite hard to properly evaluate. Optimizing compilers (and
post-compilation utilities) do a wide variety of manipulations on the
code generated for a program, or function therein. These optimizations
are very useful for most situations, but can cause problems with system
benchmarks. Optimizers may re-adjust code and totally eliminate variables,
portions of expressions, and even entire code segments. Strength reduction
may be performed on certain operations (an intended multiply operation may
be changed to a simpler multiple-addition operation). While these things
are VERY useful for other types of code, for benchmarks of systems, they
defeat the purpose. I want the system to WORK during the benchmarks, and
do EXACTLY what I intended it to do, in the manner I intended. For this
reason, heavy optimization was not used. Now, some common optimization was
done...use of register variables where possible, etc... And, a certain
amount of internal compiler optimization was unavoidable. However, I was
sure to check the output object code, and verify that I was getting a
good code model of what I wanted tested. The object of AIBB was not to
test how well a compiler could optimize code...but to see just how well
different systems could perform while operating on benchmark code.
The benchmarks employed by AIBB are meant to be used in the context
of comparing one type of Amiga, or Amiga setup to another (or to the
built-in comparison machines). The figures you receive from AIBB should
NOT be used to compare an Amiga to another architecture. The reason for
this is that, there are too many variables involved in such a comparison.
Take, for example, the Dhrystone benchmark. AIBB's implementation of the
dhrystone is "standard" (whatever that means these days), BUT...the version
on the other machine architecture may be standard, but was in compiled the
same way? Better yet, how do the compiler used for AIBB and the other
architecture differ? (Different C compiler implementations by different
authors/publishers will create different code, depending on their levels
of optimization within the compiler itself, etc...). Is the other
architecture's code further optimized? As you see, this creates a big mess
for using AIBB in such comparisons. These types of comparisons are better
left to an environment where the compiling, code generation, etc... are
all under the testing individual's control.
==============================================================================
PROGRAMMING NOTES:
AIBB was written in C, with assembly code sections where necessary.
[the Cache Control Register access routines, etc...]. It was compiled
with SAS/C 5.10a using register argument parameters for all function
calls, 16-bit PC-relative offset function call addressing, and base A4
relative data addressing. It was extensively tested on a variety of
machines, and in a variety of conditions, including AmigaOS 2.0. It should
work properly with all versions of AmigaOS, but 2.0 compatibility has not
been _thoroughly_ tested. If a problem arises under 2.0, I would appreciate
hearing about it. [In any case, as mentioned earlier, the test figures
were set for AmigaOS 1.3...in some cases the OS version will make no
difference, in others it may]. The assembly portions of AIBB were
assembled with the SAS "asm" assembler included with SAS/C 5.10a.
I want to make a comment on the use of Disable() within this program.
Disable() is a function designed primarily to allow a task to access data
structures shared by interrupt code, without worry of having this code
changed by an interrupt event during the access. My use of it here was
because it was the only real way to insure that my program would have the
best chance of being given total control of the procesessor. What I want
to make clear is this: If you are just starting to program the Amiga,
(or other multitasking systems for that matter), the use of Disable() or
functions like it is to be a last resort, unless you are doing something
very similar to what I have done here. Disable() is a very cruel thing
to do to a multitasking system, and should be avoided if at all possible.
There are some circumstances where it's use is necessary, to be sure, but
it should be considered that my use of it here is NOT a standard method
of usage for the call. [In fact, in most circumstances you should not
be Disable()'ed for more than 250ms, as per the ROM Kernel Manual
recommendations.] In the context where I have used the call for this program,
it won't cause problems provided the proper (mentioned earlier) environment
exists. But to be sure, indiscriminate calls to Disable() and such are very
poor programming practice.
==============================================================================
ACKNOWLEDGEMENTS:
I wish to thank all of the folks who helped and supported me in this
project. Especially of help was Dr J. Scott Thayer, sysop of the
Amiga Friends BBS. He spent long periods of time evaluating AIBB, and
finding bugs which I did not notice. Without his help it is doubtful
that AIBB v3.0 would have been released in any timely fashion.
==============================================================================
MISCELLANEOUS:
Questions, comments, and suggestions about this program are quite welcome.
I can be reached the following ways:
Safe Harbor BBS: Amiga Hardware Special Interest Group Operator
(SIGOP--SIG Area 5 [Amiga Hardware]) (414)548-8140
I can always be reached here.
Amiga Friends BBS: (714)870-4754 or -6954 (two lines). Generally
I can be reached here anytime as well.
AmiHolics BBS: (602)843-8486
Northwest Amiga Group (NAG): (503)656-7393
JDR Microsystems BBS: (408)559-0253
Miller's Amiga System BBS: (612)698-1485
I can also be found on USENET: lkoop@crash!pnet01.cts.com (Internet)
[ pick your path :^) ]
If you wish, you can also contact me via the U.S. Snail service at
LaMonte Koop
565 Park Meadows Dr. #302
Waite Park, MN 56387
Enjoy the program, and don't hesitate to comment on it to me. I am always
looking to improve the code I write, so I take constructive criticism
well. However, flames, useless criticism, or general nastiness get their
best attention from me by simply outputting them to either /dev/null or
NIL: ...your choice ;^).
